Cheating: its implications for American Board of Family Medicine examinees.

Cheating is undesirable and unethical, but, unfortunately, sometimes it does occur. Recent events at 3 American Board of Medical Specialties specialty boards have illustrated that the medical certification industry is not immune from this phenomenon. Although there are numerous moral and professional implications involved with cheating, we wish to address the implications of cheating from a psychometric perspective. Our intent is to highlight some of the less obvious ways in which all American Board of Family Medicine (ABFM) diplomates possibly could be impacted should those diplomates and candidates resort to cheating on examinations. So, what is cheating? Cizek defines it as “any action that violates the rules for administering a test, any behavior that gives an examinee an unfair advantage over other examinees, or any action on the part of an examinee or test administrator that decreases the accuracy of the intended inferences arising from the examinee’s test score or performance.” The ABFM goes to great length to ensure a fair test for all examinees. When examinees register for ABFM exams, they make a promise to adhere to both the ethical and legal standards associated with the administration of the examination. This pact between the ABFM and the candidates minimizes the risk of a compromised examination score(s). Unfortunately, whenmembers of either party fail to adhere to the agreed-upon standards, problems can arise. Cheating as a Threat to the Validity of Examination Scores Validity is perhaps the most important aspect of any test. The concept of validity refers to the extent to which interpretations and inferences gleaned from a score are accurate. When cheating occurs, estimates of an examinee’s performance are no longer accurate. Perhaps the most obvious example of cheating as a threat to validity occurs when an individual has an undue advantage and receives a score that is higher than his or her true estimate. The inflated score essentially would be a misrepresentation of that individual’s performance, thus yielding an inaccurate estimate of performance. More subtle ways in which cheating can affect validity exist as well. The most overt threat to examination validity would be associated with the leakage of examination items. Most testing organizations, the ABFM included, possess item banks with a large pool of items readily available for inclusion on an examination. Items often vary with regard to how many times they may be used; some items are only used once, whereas others may be used perpetually provided they remain valid from a content perspective and continue to function in a psychometrically sound manner. Some overlap of items across administrations almost always exists, although the amount of overlap varies considerably across testing organizations. In any instance, examination items that are leaked from the item bank could give those with access a significant advantage. Regardless of how the test is constructed, if a single item has been compromised it could result in some examinees receiving a score that misrepresents their actual estimates of performance. Of course, the more items that are leaked, the greater the threat to the validity of the examination. Because most high-stakes examinations are scored with some form of item response theory (IRT) methodology, the difficulty of the items plays an important role in discerning a measure of Submitted 29 February 2012; revised 29 February 2012; accepted 1 March 2012. From the American Board of Family Medicine, Lexington, KY. Funding: none. Conflict of interest: none declared. Corresponding author:Kenneth Royal, PhD, 1648McGrathiana Parkway, Suite 550, Lexington, KY 40511 (E-mail: [email protected]).